Papers with power consumption
Efficient Inference for Large Language Models –Algorithm, Model, and System (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Inference of LLMs incurs high computational costs, memory access overhead, and memory usage, leading to inefficiencies in terms of latency, throughput, power consumption, and storage. |
| Approach: | This tutorial introduces the basics of efficient inference for LLMs and explains how to diagnose efficiency bottlenecks for a given workload on specific hardware. |
| Outcome: | The tutorial introduces the basic concepts of modern LLMs, software and hardware. |
MobileNMT: Enabling Translation in 15MB and 30ms (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing work on NMT models is limited in storage, memory, computation and power consumption. |
| Approach: | They propose a mobile machine translation system that can translate in 15MB and 30ms on devices. |
| Outcome: | The proposed system can translate in 15MB and 30ms on mobile devices. |
Breaking Down Power Barriers in On-Device Streaming ASR: Insights and Solutions (2025.naacl-industry)
Copied to clipboard
Yang Li, Yuan Shangguan, Yuhao Wang, Liangzhen Lai, Ernie Chang, Changsheng Zhao, Yangyang Shi, Vikas Chandra
| Challenge: | Streaming automatic speech recognition models use high power consumption to improve usability and accuracy. |
| Approach: | They propose to optimize on-device speech recognition models by adjusting component energy sensitivities based on their specific energy sensitities to reduce power consumption. |
| Outcome: | The proposed approach achieves up to 47% lower energy usage while preserving comparable model accuracy and improving real-time performance compared to leading methods. |
ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yash Akhauri, Ahmed AbouElhamayed, Jordan Dotzel, Zhiru Zhang, Alexander Rush, Safeen Huda, Mohamed Abdelfattah
| Challenge: | Prior work has focused on contextual sparsity, but it has not been successful. |
| Approach: | They propose a novel pruning predictor that can shadow the LLM behavior and enforce better sparsity patterns. |
| Outcome: | The proposed model can shadow the LLM behavior and enforce better sparsity patterns, resulting in 15% improvement in end-to-end accuracy compared to prior methods. |
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) fine-tuning techniques require large Floating Point(FP) computation and are impractical for resource-constrained edge devices. |
| Approach: | They propose a framework for on-device LLM fine-tuning that eliminates the need for floating-point operations in both inference and training. |
| Outcome: | The proposed framework reduces memory and compute costs while reducing memory usage. |